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ABSTRACT 

In 1985-86, after five years of implementation, the 
New York City (New York) Public Schools transferred responsibility 
for the Promotional Policy Program to the community school district. 
This report provides information abodt the success of the remedial 
reading program under this new organization in terms of student 
outcomes. Promotional "gates" in grades 4 and 7 were devised in 1980, 
with a students' promotion dependent upon meeting the criterioi for 
that grade on the city-wide reading test. Students who failed to meet 
the criterion set on the California Achievement Test in reading or 
(for eligible limited English proficient students) on the Language 
Assessment Battery were eligible for the program. A total of 13,73** 
fourth- and seventh-grade students was eligible. Of the 12,248 
students with both pre- and post-test scores, 81.5% met the criterion 
for their grade in August of 1985, January of 1986, or April of 1986. 
However, average gains were less than those made by students who 
participated in previous years. Patterns of achievement by 
promotional category were different from those of prior years. 
Achievement of students in an extension program continues to be 
problematic; gains of eighth-grade extension students, particularly, 
do not match those of other program students. It is re'^ommended that 
the central and district program administrators review the programs 
for adequacy of teacher preparation and curriculum materials. 
Thirteen data tables are included. (TJH) 
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A SUMMARY OF THE EVALUATION OF THE 
STUDENT ACHIEVEMENT IN THE 1985-86 
PROMOTIONAL POLICY PROGRAM* 

The Promotional Policy 

The promotional policy of the New York City Board of 
Education was established to set and maintain city-wide curricu- 
lum and performance standards, to identify students unable to" 
meet these standards, and to provide remedial instruction for the 
Identified students. Introduced in June, 1980, the policy de- 
fined promotional "gates" in grades four and seven, with a stu- 
dent s promotion being dependent upon meeting the criterion for 
that grade on the city-wide reading test. By instituting uni- 
form, obaective promotional standards, the program established 
specific expectations for students and goals for staff. 

Equitable and formalized procedures for granting exceptions 
were established m the second year of the program. These re- 
duced the program's reliance on a single test score for determin- 
ing eligibilty. The program also provides multiple opportunities 
for students to demonstrate criterion attainment through special 
testing in August and January as well as regular test administra- 
tion m April. 

In 1985-86, after five years of implementation, central ad- 
ministrators gave responsibility for ongoing staff development 
and supervision of this program to the community school dis- 
tricts. This report provides information about the success of 
the program under this new organization in 1983 86 in terms of 
student outcomes. 

Gates-Eligible Students 

A total of 13,734 students who were in fourth and seventh 
grade for the first time in 1984-85 failed to meet the criterion 
set on the California Achievement Test (CAT) in reading or (if an 
eligible limited-English-prof icient student) on the Language 
Assessment Battery (LAB). These students were eligible for 
placement m Gates classes in 1985-86. An additional 620 stu- 
dents who had been in fourth-grade Gates classes and 864 students 
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*This summary is based on "Student Achievement in the 1985-86 
Promotional Policy Program" (July, 1987) prepared by the o.E.A. 
Student Progress Evaluation Unit. 
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who had been in seventh-grade Gates classes in 1984-85 had April, 
1985 test scores below criterion. These students were promoted 
but were eligible for Extension classes. 

Achievem e nt of Gates Students 

Of the 12,248 Gates students with both pretest and posttest 
scores, 81.5 percent met the criterion for their grade in August, 
1985; January, 1986; or April, 1986. However, both the average 
fourth- and average seventh-grade gains were less than those made 
by Gates students in previous years. Approximately 85 percent of 
the Gates population also had scores on the Degrees of Reading 
Power (D.R.P.) Test in April, 1985. Comparison of the April, 
1985 and April, 1986 D.R.P. scores indicated educationally 
significant gains. 

Over the five years of the program, 78.8 percent of the 
fourth-grade and / 9 percent of the seventh-grade Gates students 
attained their read .g criterion. The average full-year gain 
over the five years nad been consistent in the fourth and seventh 
grades until 1985-86 when the gains were lower. 

Patterns of achievement of Gates students by promotional 
category were different from those of prior years. In prior 
years, students promoted in April had lower gains than those pro- 
moted in August or January. Students promoted in April, 1986, 
however, made Normal Curve Equivalent score gains equivalent to 
students promoted in August and January. Those students who did 
not meet criterion and became Extension-eligible students made 
negligible gains. Additional analyses of full-year gains based 
solely on the D.R.P. scores indicated similar patterns of 
achievement. 

Patterns of achievement by subgroups of the Gates population 
are similar to those of previous years in that the average 
achievement of Resource Room students in 1985-86 was slightly 
less than that of the full Gates population in both fourth and 
seventh grade. As in prior years, LEP students' gains were less 
than those of the entire group in both grades. 

Achiev ement of Extension Stiid^nhg 

Achievement of Extension students continues to be problema- 
tic. The gains of eighth-grade Extension students, particularly, 
do not match those of Gates students overall. However, analyses 
using April, 1985 and April, 1986 D.R.P. scores indicated educa- 
tionally significant gains for both the fifth- and eighth-grade 
Extension students. 



C onclusions and Recoimnendation s 

Evaluation of the Promotional Policy Program over the past 
five years suggests that the program continues to have a consis- 
tently positive effect on academic achievement. In 1985-86 
however, the percent of fourth- and seventh-grade students ' 
attaining their criterion increased yhile the gains decreased 
The criterion attainment results may be an artifact of new test- 
ing procedures; specifically, the use of two tests (the d R P 
and the MAT) to determine criterion attainment. The decrease* in 
gams, therefore, is problematic. Based on these findings, 
O.E.A. recommends that central and district program administra- 
tors review the program for changes that may have affected pro- 
gram outcomes. 



ERIC 



Evaluation Section Report 
RoDert ToDias, Aaministrator 
Pruaence Opperman, Senior Mancger 



July, 1987 



STUDENT ACHIEVEMENT IN 
THE 1985-86 
PROMOTIONAL POLICY PROGRAM 



Preparea by the O.E.A. 

Student Progress Evaluation Unit 



Henry Solomon 
Evaluation Manager 

Richard D. Gampert 
Evaluation Specialist 

Augustus Olu-Hamilton 
Educational Analyst 



New York City Public Schools 
Office of Educational Assessment 
Richard GuttenDerg, Director 



It is the policy of the Board of Education not to discriminate on the basis of race, creed, national origin, age, 
handicapping condition, sexual orientation, or i^x, in Its educational programs, activities, and employment policies 
as required by law. Any person who believes he or she has been discriminated against should contact; Carole Guerra, 
Local Equal Opportunity Coordinator, Office of Educational Assessment, 110 Livingston Street, Room 743, Brooklyn, 
New York 11201. Inquiries regarding compliance with appropriate laws may also be directed to: Mercedes A. Nesfield, 
Director, Office of Equal Opportunity, 110 Livingston Street, Room 601, Brooklyn, New York; or the Director, Office 
of Civil Rights, U.S. Department of Education, 26 Federal Plaza, Room 33-130, New York, New York 10278 



7 



A SUMMARY OF THE REPORT 



The Promotional Policy 

The promotional policy of the New York City Board of Education was 
established to set and maintain city-wide curriculum and performance 
standards, to identify students unable to meet these standards, and to 
provide remedial instruction for the identified students. Introduced in 
June, 1980, the policy defined promotional "gates" in grades four and 
seven, with a student's promotion being dependent upon meeting the 
criterion for that grade on the city-wide reading test. By instituting 
uniform, objective promotional standards, the program established specific 
expectations for students and goals for staff. 

Equitable and formalized procedures for granting exceptions were 
established in the second year of the program. These reduced the 
program's reliance on a single test score for determining eligibilty. The 
program also provides multiple opportunities for students to demonstrate 
criterion attainment through special testing in August and January as well 
as regular test administration in April. 

In 1985-86, after five years of implementation, central administrators 
gave responsibility for ongoing staff development and supervision of this 
program to the community school districts. This report provides infor- 
mation about the success of the program under this new organization in 
1985-86 in terms of student outcomes. 

Gates-Eligible Students 

A total of 13,734 students who were in fourth and seventh grade for the 
first time in 1984-85 failed to meet the criterion set on the California 
Achievement Test (CAT) in reading or (if an eligible limited-English- 
proficient student) on the Language Assessment Battery (LAB). These 
students were eligible for placement in Gates classes in 1985-86. An 
additional 620 students who had been in fourth-grade Gates classes and 864 
students who had been in sevonth-grade Gates classes in 1984-85 had April, 
1985 test scores below criterion. These students were promoted but were 
eligible for Extension classes. 

Achievement of Gates Students 

Of the 12,248 Gates students with both pretest and posttest scores, 81.5 
percent met the criterion for their grade in August, 1985; January, 1986; 
or April, 1986. However, both the average fourth- and average seventh- 
grade gains were less than those made by Gates students in previous years. 
Approximately 85 percent of the Gates population also had scores on the 
Degrees of Reading Power (D.R.P.) Test In April, 1985. Comparison of the 
April, 1985 and April, 1986 D.R.P. scores indicated educationally 
significant gains. 
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Over the five years of the program, 73. 8 percent of the fourth-grade and 
72.9 percent of the seventh-grade Gates students attained their reading 
criterion. The average full-year gain over the five years had been con- 
sistent in the fourth and seventh grades until 1985-86 when the gains were 
lower. 

Patterns of achievement of Gates students by promotional category were 
different from those of prior years. In prior years, students promoted in 
April had lower gains than those promoted in August or January. Students 
promoted in April, 1986, however, made Normal Curve Equivalent score gains 
equivalent to students promoted in August and January. Those students who 
did not meet criterion and became Extension-eligible students made negli- 
gible gains. Additional analyses of full-year gains based solely on the 
D.R.P. scores indicated similar patterns of achievement. 

Patterns of achievement by subgroups of the Gates population are similar 
to those of previous years in that the average achievement of Resource Room 
students in 1985-86 was slightly less than. that of the full Gates popula- 
tion in both fourth and seventh grade. As in prior years, LEP students' 
gains v^ere less than those of the entire group in both grades. 

Achievement of Extension Students 

Achievement of Extension students continues to be problematic. The 
gains of eighth-grade Extension students, particularly, do not match those 
of Gates students overall. However, analyses using April, 1985 and April, 
1986 D.R.P. scores indicated educationally significant gai for both the 
fifth- and eighth-grade Extension students. 

Conclusions and Recommendations 

Evaluation of the Promotional Policy Program over the past five years 
suggests that the program continues to have a consistently positive effect 
on acadernic achievement. In 1985-86, however, the percent of fourth- and 
seventh-grade students attaining their criterion increased while the gains 
decreased. The criterion attainment results may be an artifact of new 
testing procedures; specifically, the use of two tests (the D.R.P. and the 
MAT) to determine- criterion attainment. The decrease in gains, therefore, 
is problematic. Based on these findings, O.E.A. recommends that central 
and district program administrators review the program ^or changes that may 
have affected program outcomes. 
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I . INTRODUCTION 

BACKGROUND 

In June, 1980, the promotional policy of the New York City Board of 
Education was established to set and maintain city-wide curriculum and 
performance standards, identify students unable to meet these standards, 
and provide remedial instruction for the identified students. The policy 
continues to maintain promotional "gates" in grades four and seven, with a 
student's promotion dependent upon meeting the criterion set for that grade 
on the city-wide reading test. By instituting uniform, objective pro- 
motional standards, the program establishes specific expectations for 
students and goals for staff. 

Procedures for granting exception^ established in the second year of 
the program, continue to reduce the program's reliance on a single test 
score for determining eligibility. (See Final Evaluation of the 1981-8? 
Promotional Gates Program and A Final Evaluation of the 1982-83 
Promotional Policy Pro.gram.) The program also provides multiple oppor- 
tunities for students to demonstrate criterion attainment through special 
test administrations in August and January as well as the regular test 
administration in April, 

In 1985-36, after five years of implementation, central administrators 
git- Wilbility for ongoing staff development and supervision of the 
P the community school districts. This report provides information 

about success of the program in terms of student outcomes. 
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DESCRIPTION OF THE DATABASE 

As in previous years, the analysis of test results in the fifth year 
of the program utilized information supplied by Student Information 
Systems (SIS). This database consists of test records of all students 
who were in fourth or seventh grade in April, 1985. 

A total of 13,734 students (see Appendix A, Table A-1) were identi- 
fied who were in either the fourth or seventh grade for the first time 
and whose April, August, or September, 1985 reading score was below 
criterion for their grade on the California Achievement Te- ^t (CAT). 
Limited English Proficient (LEP) students who had been in an English 
language school system for at least two but less than four years were 
identified from data entered on the Language Assessment Battery (LAB) 
test answer document. These LEP students were subject to the LAB promo- 
tional criteria;* all other non-exempt students were subject to tne 
reading criteria on the CAT. 

In addition, 620 students who had been in the 1984-85 fourth-grade 
Gates program and 864 students who had been in the 1984-85 seventh-grade 
Gates program had April, 1985 test scores that were still below crite^^ion. 
These students were eligible for the 1985-86 fifth-grade and eighth- 
grade Extension programs, respectively. 

This report evaluates the progress in 1985-86 of these Gates- and 
Extension-eligible students through analyses of standardized reading 
achievement test scores. 

*The criterion for fourth grade was a total test raw score of 47 or 
above. For seventh grade, the criterion was a reading subtest raw 
score of 24 or above. 

-2- 
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City-Wide Achievement Tests 

For April, 1985 through January, 1986, as in previous years, the 
city-wide reading achievement test was the CAT. The fourth grade cri- 
terion on the CAT i as the 27th percentile (3.7 grade equivalent) for the 
spring, summer, and fall testing. For the January testing the cri- 
terion was the 46th percentile (4'.2 grade equivalent). In the seventh 
grade, the criterion was the 30th percentile (6.2 grade equivalent) for 
the spring, summer, and fall testings and the 43rd percentile (7.2 grade 
equivalent) for the January testing. 

In April, 1986 two tests were used: the primary test was the 
Degrees of Reading Power (D.R.P.) and the secondary test was the 
Metropolitan Achievement Test (MAT). The criterion set for promotion on 
the D.R.P. was the 32nd percentile (based on national norms) for both 
the fourth and seventh grades. The criterion set for promotion on the 
MAT was the 16th percentile for fourth grade and the 15th percentile for 
seventh grade. The apparent differences in these criteria are the re- 
sult of inherent psychometric differences between the D.R.P. and MAT. 
DATA ANALYSIS 

The CAT and D.R.P. results could not be directly compared (i.e., 
scores on one test could not be transformed into scores on the other 
test). Therefore, in order to assess changes in the performance of the 
Gates and Extension students, scores on both tests were converted to 
their Normal Curve Equivalents (N.C.E.s). The N.C.E.s were then used to 
assess changes in performance. This procedure produced results that are 
different from, but not inconsistent with, results obtained in 



previous years. These differences will be discussed as they arise in 
later sections of this report. 

The primary function of the D.R.P. is to identify the level of 
reading materials students can handle independently and during instruc- 
tion. The D.R.P. mid-instructional score, used in some of the follow- 
ing analyses, provides an estimate of the level of materials a student 
will be able to understand well enough to learn from during classroom 
instruction. 

OVERVIEW OF THE REPORT 

Chapter II of this report presents results by Gates students of 
promotional criteria by Gates students in grades four and seven. The 
results of analyses on gains in reading achievement are also presented. 
O.E.A. also analyzed the reading achievement of various subgroups of 
the Gates population, specifically resource room and LEP-eligible 
students. 

Chapter III of this report presents the conclusions and recommenda- 
tions made by O.E.A. after analyzing and reviewing the results. 
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II. STUDENT ACHIEVEMENT 
ATTAINMENT OF PROMOTIONAL CRITERIA BY GATES STUDENT<^ 
Description of the Gates Popu1ati--n 

According to the records of the Office of Promotional Policy (O.P.P.), 
7.5 percent of the total fourth grade and 8.6 percent of the total seventh 
grade were Gates-eligible in 19C5-86. These 1985-86 percentages are up 
slightly from those of 1984-85, but they are below the percentages observed 
in 1981-82, 1982-83, and 1983-84. Among the 1935-86 students originally 
held over, slightly more than 10 percent of the fourth and seventh graders 
were granted exceptions; the percent of exceptions granted has been 
steadily increasing each year of the program. (See Table A-1 in Appendix 
A.) CP. P. records indicate that, overall, only 61 percent of eligible ' 
students were placed in promotional policy classes in 1985-86; this is a 
substantial decline in placememt from prior years. 
1985-86 Criterion Attainment 

Of the 13,734 Gates students, 12,248 had both pretest scores on the CAT 
and posttest scores on the D.R.P. Of these students, 9,979 (81.5 percent) 
met the promotional criterion for their grade on one of the three dates 
when the reading tests were administered: August, 1985; January, 1986; or 
April, 1986. Unlike previous years, a greater proportion of the seventh 
than fourth graders (83.6 and 79.0 percent, resoectively) met the criterion 
for their grade. A total of 2,269 students (18. b percent) in the combined 
grades were unable to attain criterion. (See Table 1.) These students 
became eligible for the 1986-87 Extension program. 

-5- 
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TABLE 1 

Comparison of Reaamg Criterion Attainment of 
l981-82» 1982-83, 1983-84, 1984-85, ano 1985-86 Gates Students^ 



1981-82 



1982-83 



1983-84 
"H T 



1984-85 



1985-86 



I 
I 



Grade Four 

Students with pretest and 

posttest scores 8,434 

Students meeting criterion 

August 2,124 25.2 

January 488 5.8 

April 3.884 46.0 

Total meeting criterion 6,496 77.0 

Extension ellglDles 1,938 23.0 
Grade Seven 

Students with pretest and 

postt:!St scores 10,219 

Students meeting criterion 

August 2,548 24.9 

January 1,284 12.6 

April 2.642 25.9 

Total meeting criterion 6,474 63.4 

Extension ellglDles 3,745 36.6 



6,808 

1,920 
376 
3,140 

5,436 

u372 



9,402 

3,357 
970 
2,755 

7,082 

2,320 



28.2 
5.5 
46.1 

79.8 

20.2* 



35.7 
10.3 
29.3 

75.3 

24.7 



5.127 

1,537 
• 300 
2,227 

4,064 

1,063 



7,799 

2,208 
1,076 
2.100 

5,334 

2,415 



30.0 
5.9 
43.4 

79.3 

20.7 



28.3 
13.8 
26 >9 

69.0 

31.0 



6,339 

1,505 
330 
3,208 

5,043 

1,296 



8,379 

2,832 
811 
2,811 

6,454 

1,925 



23.7 
5.2 

50.6 

79.5 
20.5 



33.8 
9.7 
33.5 

77.0 

23.0 



5,62: 

1,588 
277 
2,578 

4,443 

1,184 



6,621 

2,088 
842 
3,606 

5,536 

1,085 



28.2 
'.9 
45.8 

79.0 

21.0 



31.5 
12.7 
39.4 

83.6 

16.4 



^I'^^Jnof^iS ^JoJU/eadlng-only and reading and/or mathematics Gates-eligible students in 1982-83 

m 1981-82, 1983-84, 1984-85, and 1985-36 students were held over If they were Dslow Se readlSg crU 
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Comparison of Reading Criterion Attainment 



Attainment of reading criterion in 1985-86 can be compared with at- 
tainment in previous years. Table 1 includes data for the first five 
Gates school years. In the fourth grade, the rate of attainment in 
1985-86 (79.0 percent) is comparable to the rates in the previous years. 
In the seventh grade, 1985-86 criterion attainment (83.6 percent) was 
greater than in any previous year. This difference occurs at the April, 
1986 testing, at which time students could attain their grade criterion 
on either the D.R.P. or the MAT. Overall, 78.8 percent of the fourth- 
grade Gates students and 72.9 percent of the seventh-grade Gates stu- 
dents attained the reading criteria in the five years of the program. 

READING ACHIEVEMENT GAINS OF GATES STUDENTS ON THE CAT AND D.R.P. 
Overall Achievement 

The achievement test scores of Gates students who had both an April, 
1985 pretest score on the CAT and an April, 1986 posttest score on the 
D.R.P. were analyzed for gains. The results of these analyses appear in 
Table 2. The scores are expressed in Normal Curve Equivalent (N.C.E.) 
units since this allows comparisons of scores on the CAT with scores on 
the D.R.P. Pretest means were adjusted to account for regression to the 
mean.* Regression to the mean is a statistical artifact which results 
from using the same test for both pupil selection and as the pretest in 
program evaluation, especially in populations which have distributions 
that differ from the norming group. 

* Only April to April scores of the total test population can be adjusted 
for the regression effect. Refer to Appendix B for these computations. 
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TABLE 2 



Reading Achievement of Gates Students 
on the Citywide Tests 



Grade 



Pretest N.C.E.^ 



Unadjusted Adjusted 
Mean Mean 
N.C.E. (S.D.) N.C.E. 



Adjusted 
Mean 

Posttest N.C.E.'' Difference 



K^SIe. (S.D. 



) 



Four 4,998 
Seven 5,750 



28.8 (8.0) 34.0 

31.9 (7.4) 36.0 



44.3 (13.9) 10.3 
42.8 ( 9.9) 6.8 



^The pretest was the CAT, given as the citywide reading test in April, 1985. 

^The posttest was the D.R.P., given as the citywide reading test in April, 1986. 

CAn additional 832 fourth- and 1,350 seventh-grade students were eligible for the 
Gates program on the basis of a below-criterion reading achievement score. These 
students were missing one or the other April score and could not ba included in 
these gains analyses. 
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The data analysis results, by grade, indicate that Gates students 
made progress in reading during the 1985-86 school year. Fourth-grade 
students gained an average of 10 N.C.E.s and seventh-grade students 
gained an average of seven N.C.E.s. 

Comparison of Gains Across Five School Years 

During the first four years of the program, gains of Gates students 
were contrasted with those of a "comparison group." Comparison group 
students were 1980-81 students who would have been eligible for Gates if 
the program had existed that year. This comparison was repeated to 
include the current fifth-year results. The data are found in Table 3. 
The data in Table 3 are reported in N.C.E.s to provide direct compari- 
sons between previous years and the current year of the program. While 
the 1985-86 Gates students have slightly higher pretest scores than 
students in previous years, these differences are minimal. An analysis 
of covariance, therefore, was not deemed necessary since any adjustment 
in the posttest scores would be minimial. Nevertheless, gains of Gates 
students in 1985-86 appear to be less than in prio.r years of the program. 
Reading Achievement by Promotional Category 

Table 4 presents the results of analyses on the full-year gains made 
by students who were eligible for the Gates program for different 
lengths of time: students promoted in August, 1985; students promoted 
in January, 1986; and full -year holdovers. 

Students who met criterion in August, 1985 . The promotional policy 
allowed students who scored below criterion on the April, 1985 CAT to 
take the test again at the end of summer school. Full-year gains of 
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TABLE 3 



Comparison of Reading Ac hi evement on the Citywide Tests of the 
1980-81 Comparison Gr up ancJ 1981-82, 1982-83, and 1983-84, 
1984-85, and 198L'-86 Gates Students 



Observed mean Observed mean 

School pretest posttest 

year N N.C.E. score N.C.E. score Difference 



Grade Four 










1980-81 


6,914 


27 


42 


15 


1981-82 


6,924 


27 


44 


17 


1982-83 


6,003 


27 


44 


17 


1983-84 


4,226 


28 


44 


16 


1984-85 


5,741 


27 


44 


17 


1985-86 


4,998 


29 


44 


15 


Grade Seven 










1980-81 


10,214 


30 


42 


12 


1981-82 


8,659 


29 


41 


12 


1982-83 


8,439 


30 


44 


14 


1983-84 


6,230 


31 


44 


13 


1984-85 


7,471 


31 


45 


14 


1985-86 


5,750 


32 


43 


11 



NOTE: Mean posttest scores are not adjusted for pretest level, and no covari- 
ance analysis for the five years has been performed. In addition, 
these are observed scores which have not been adjusted for regression 
and as a result overestimate actual gains. 



-10- 



ERIC 



Z2 



TABLE 4 



Reading Achievement of Gates Students by Promotional Category 



Pretest^ 


Posttest'^ 




Mean 
N.C.E. 

Difference 


Grade N Mean (S.D.) 

N.C.E. 


Mean 
N.C.E. 


(S.D.) 




Students who met criterion in August, 1985 








Fouv' 1,492 30.3 (5.9) 


46.5 


(10.6) 


16.2 


Seven 1,925 33.2 (5.5) 


45.7 


( 8.0) 


12.5 


Students who met criterion in January, 1986 








Four 236 29.8 (6.7) 


49.7 


(11.1) 


19.9 


Seven 717 32.7 (6.0) 


43.9 


( 8.7) 


11.2 


Students who met criterion in April , 1986 








Four 2,356 29.0 (8.6) 


49.7 


(10.9) 


20.7 


Seven 2,416 32.2 (7.8) 


44.8 


( 7.4) 


12.6 


Extension-eligible studeiV;s 








Four 914 25.5 (8.7) 


25.1 


( 8.4) 


-0.4 


Seven 692 26.6 (9.3) 


26.9 


( 8.8) 


0.3 



^ The pretest was the CAT, given as the citywide reading test in April, 1985. 

^ The posttest was the D.R.P., given as the citywide reading test in April, 1986. 

^ These observed mean N.C.E. differences, which for subgroups cannot be adjusted for 
the regression effect, overestimate actual gains. 
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students who took and passed the August test appear in Table 4. These 
students were eligible for promotion to a 1985-86 fifth- or eighth-grade 
class. Subsequently, fourth grade students in this group gained over 16 
N.C.E.s on the April, 1986 D.R.P. This level of performance is compar- 
able to the gains achieved in previous years. Seventh-grade gains were 
less than previous years (12.5 N.C.E.s). 

Students who met criterion in January. 1986 . In January, the cri- 
teria for promotion were higher. These criteria were increased for 
January so that students promoted at mid-year would be more likely to 
perform successfully in the higher grade and, thus, would be better able 
to attain the end-of-year promotional criterion for their new grade. 
These students were eligible for participation in a Gates class for the 
first half of the 1985-86 school year, and for placement in a regular 
fifth- or eighth- grade class for the remainder of the year. In the 
fourth grade, the average April, 1986 score was 20 N.C.E.s higher than 
the April, 1985 score. In the seventh grade the average score was 11 
N.C.E.s higher than the April, 1985 score. 

Full -Year Holdovers . Table 4 also presents gains in reading achieve- 
ment made by full-year holdovers. This group includes those students 
who had not met the criterion in either August, 1985 or January, 1986. 
The full -year holdover group is divided into two subgroups: those who 
met the criterion in April, 1986 and those who did not. In both grades 
the e^yerage pretest score of students who met criterion in April was 
equivalent to that of students who met their criterion at an earlier 
point. Also, the average gain in reading achievement was equivalent to 
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or higher than that of students who met the criterion earlier. On the 
other hand, students who did not meet the criterion for their grade and 
became Extension-eligible students did not demonstrate gains. As has 
been noted in prior reports, the gains of this group are difficult to 
measure since the pretest and posttest scores are near chance Itvel. 
Random guessing could produce scores in this range. Therefore, the 
gains of these students cannot be measured ren'ably. Nevertheless, the 
results indicate that these students continue to perform below the 
promotional standard. 

Reading Achievement by Subgroups of the Gates Population 

Resource Ro om Students . Table 5 reports gains in reading achieve- 
ment by Gates students who were also in a special -education Resource 
Room. Their performance on the reading achievement tests was similar to 
that of the total Gates population (of which they are a part). In each 
grade the average pretest and posttest scores were slightly lower than 
that of the full group and their N.C.E. gains were slightly less than 
those gains made by the total population. 

Limited English Proficient (LEP) Students* . LEP students who had 
been in an English language school system for four years or more were 
required to meet the promotional criteria on the CAT in April, 1985 
and the D.R.P. in April, 1986. Table 6 shows the results of the analysis 
of the gains these students made over the course of the year. As in the 
previous years of the program, LEP students had lower pretest and 

* The operational definition of limited English proficiency (LEP) is a 
score at or below the twentieth percentile on the English reading 
subtest of the LAB. 
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TABLE 5 

Reading Achievement of Resource Room Students 









Pretest^ 




Posttest'' 




Mean N.C.E. 










D.R.P. 




Difference 


Grade 


N 


Mean 


(S.D.) 


Mean 


(S.D.) 








N.C.E. 




N.C.E. 




Four 


284 


27.3 


(7.6) 


41.6 


(14.3) 


14.3 


Seven 


225 


30.6 


(7.3) 


41.1 


( 8.3) 


10.5 



3 The pretest was the CAT, given as the citywide reading test in April, 1985. 
^ The posttest was the D.R.P. , given as the citywide reading test in April, 1986. 

TABLE 6 

Reading Achievement of LEP-Eligible Gates Students 



Grade 



Pretest 



Mean 
N.C.E. 



(S.D.) 



Four 491 26.6 
Seven 430 28.7 



(8.1) 
(8.7) 



Pretest 



Mean 
N.C.E. 



(S.D.) 



39.4 
37.1 



(14.2) 
(10.5) 



Mean N.C.E. 
Di f f erence 



12.8 
8.5 



a The pretest was the CAT, given as the citywide reading test in April, 1985. 
^ The posttest was the D.R.P., given as the citywide reading test in April, 1986. 
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posttest scores than those of the total Gates population. Also, the 
gains of both the fourth-grade and the seventh-grade LEP students were 
not as great as for the total fourthand seventh -grade Gates population. 

LEP Studen ts un the Language Assessment Battery (LAB) . LEP students 
who were in an English language school system for at least two but less 
than four years were required to meet the promotional criterion on the 
LAB. Since Spring, 1984, fourth-grade LAB-eligible students hsve been 
required to meet a criterion of a total test r^tw score of 47 on the 
April LAB. The criterion for eventh-grade LAB-eligible students is a 
reading subtest raw score of 24 or above. 

Table 7 displays the results for the fourth and seventh grade of the ■ 
analysis of gains on the LAB. In the fourth-grade, the average LAB- 
eligible student obtained a raw score above the LAB promotional criterion 
(57.0) by the end of the program year. In the seventh-grade, the 
average raw score (26.6) was slightly above the promotional criterion at 
the end of the program year. 

READING ACHIEVEMENT GAINS OF GATES STUDENTS ON THE D.R.P. 

In addition to the CAT, a large percentage (85.5 percent) of the 
Gates students took the D.R.P. during the Spring, 1985 testing period. 
At that time, the D.R.P. was administered as a secondary test, the 
results of which were used to determine program exceptions. The achieve- 
ment of the 8,393 Gates students who had both pretest and posttest 
D.R.P. scores is found in Table 8. These results Indicate that Gates 
students in both grades four and seven showed educationally significant 
gains. (The D.R.P. manual states that a gain of four D.R.P. units or 
better is considereJ educationally significant.) 
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TABLE 7 



Reaaing Achievement on the LAB of LEP-Eligible Gates Stuaents 







Pretest 


Posttest 


Difference 


Graae 


\i 


Mean 

Raw 

Score 


(S.D.) 


Mean 

Raw 

Score 


(S.D.) 


Raw 
Score ' 


Four 


68 


39.9 


(8.3) 


57.0 


(12.6) 


17.1 


Seven 


42 


17.3 


(3.4) 


26.6 


( 9.0) 


9.3 








TABLE 


8 










Reaaing Achievement On The 


D.R.P. of Gates Stuaents 










Pretest 




Posttest 




Difference 






Mia 




Mia 










Instructional 


Instructional 






Graae 


N 


Score 


(S.D,) 


Score^ 


(S.D.) 




Four 


3,953 


29.2 


( 7.4) 


37.8 


( 9.8) 


8.6 


Seven 


4,440 


47.7 


( 7.1) 


53.5 


( 6.7) 


5.8 


^ Scores are expressed in D.R.P. units. 
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Reading Achievement by Promotional Category 

Table 9 presents the full -year gains on the D.R.P. made by students 
eligible for the Gates program for different lengths of time. Students 
promoted in January, 1986 had the highest full -year gains; the gains of 
students who met their promotional standard in April were higher than 
those of students promoted in August. Further, fourth-grade Gates 
students had higher full-year gains than the seventh-grade Gate^ stu- 
dents. ^The results in Table 9 closely resemble those for the CAT-D.R.P. 
analyses reported in Table 4. 

The full -year holdover group was divided into those attaining their 
grade criterion and those failing to attain their grade criterion. These 
results, also found in Table 9, indicate that students attaining their 
grade criterion in April, 1986 made educationally significant gains 
based on their D.R.P. scores. Students failing to attain their criterion 
made minimal gains on the D.R.P. Gains in the magnitude observed for 
the Extension-eligible students are not statistically significant and, 
therefore, could be attributed l.o chance. 

THE FIFTH-GRADE AND EI ^- HTH-GRADE EXTENSION PROGRAMS 

Fourth-grade and seventh-grade students who had been eligible for 
Gates Program placement in 1984-85 and who failed to attain the reading 
criterion in April, 1985 were eligible for placement in the 1985-86 
fifth- and eighth-grade Extension programs, respectively. The criterion 
attainment and achievement gains made in reading by these students are 
discussed below. 
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Table 9 



Reading Achievement On The D.R.P. of Gates Students 
By Promotional Category 



Pretest 






Posttest 




Difference 


Mid 

Instructional 
Grade N Score 


(S.D.) 


Mid 

Instructional 
Score 


f \ 




Students promoted in August, 1985 












Four 1,213 32-7 


(. 


6.5) 


41.5 


( 8.9) 


8.8 


Seven 1,561 50.3 


( 


6.1) 


1 




U .o 


students promoted in January, 1986 












Four 201 30.4 


( 


6.7) 


42.8 


( 9.0) 


12.4 


Seven 555 47.8 


( 


5.7) 


54.8 


( 5.3) 


7.0 


April, 1986 promotees 












Four 1,866 29.2 


( 


6.7) 


39.8 


( 7.4) 


10.6 


Seven 1,830 47.5 


( 


6.7) 


53.9 


( 5.3) 


6.4 


Extension-eligible students 












Four 673 22.4 


{ 


5.9) 


24.3 


( 5.0) 


1.9 


Seven 494 40.0 


( 


7.0) 


42.1 


( 5.4) 


2.1 
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Reading Criterion Attainment 

Of the 620 fifth-grade Extension students, 561 had a pretest and 
posttest score on the CAT and D.R.P., respectively. (See Table 10.) Of 
those students with complete test records, 349 (62 percent) met the 
reading promotional criterion in April, 1986 but 212 (37 ♦S percent) were 
unable to attain their criterion score on the D»R»P» and, as a result, 
did not meet the end-of-year promotional standard for a third consecutive 
year. Of the 864 eighth-grade Extension students, 694 had both pretest 
and posttest scores. A total of 460 (66.3 percent) met the promotional 
criterion by April, 1986 while 234 (33.7 percent) did not. 

Achievement on the CAT and D.R.P . The test results for 535 of the 
fifth-grade Extension students and 630 of the eighth-grade Extension 
students who had both pretest and posttest scores were analysed for 
gains. The results for these data appear in Table 11. As in prior 
years, the average N.C.E. gains for the fifth-grade Extension students 
were substantially lower than gains for the fourth-grade Gates population. 
Similar results were observed for the eighth-grade extension students in 
comparison to the seventh-grade Gates population. 

Achievement on the D.R.P . Results of the 398 fifth-grade and 483 
eighth-grade Extension students who had both April, 1985 and April, 1986 
D.R.P. scores are found in Table 12. The full-year gains of the 
Extension students are equivalent to those of the Gates population and 
are educationally significant. 
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TABLE 10 



Reading Criterion Attainment of Fifth-Grade and Eighth-Grade Extension Students 

Fifth-Grade Extension Eighth-Grade Extension 

H % H % 



Students with pretest 

and posttest scores* 561 694 

Students meeting criterion - 

August, 3985 28 5.0 28 4.0 

Students meeting criterion - 

Januory, 1986 2 0.3 5 0.8 

Students meeting criterion - 

April, 1986 319 56.9 427 61.5 

Total meeting criterion 349 62.2 460 66.3 

Below reading criterion 

three times 212 37.8 234 33.7 



* Fifty-nine fifth-grade and 170 eighth-grade students lacked posttest scores. 
These students were not included in the analysis. 
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TABLE 11 



Reaaing Achievement of Fifth-Graae ana Eighth-Graae 
Extension Stuaents 



Pretest^ Posttest^ Difference 

Normal Normal 

Curve Equivalent Curve Equivalent 

Graae N Score (S.D.) Score (S.D.) 

Five 536 28.1 (7.7) 37.8 (14.4) 9.7 

Eight 646 30.4 (7.5) 35.9 (11.1) 5.5 

^ The pretest was the CAT, given as the citywiae reaaing test in April, 1985. 

^ The posttest was the D.R.P., given as the citywiae reaaing test in April, 1986. 



TABLE 12 



Reaaing Achievement On The D.R.P. 
of Fifth-Graae ana Eight-Graae Extension Stuaents 



Pretest 



Posttest 



Difference 



Graae 



MTd 

Instructional 
Score^ 



(S.D.) 



Mia 

Instructional 
Score^ 



(S.D.) 



Five 398 28.3 ( 7.3) 35.4 ( 9.1) 7.1 

Eight 483 44.6 ( 7.1) 50.2 ( 6.6) 5.6 

3 In D.R.P. units. 
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III. CONCLUSIONS 
ACHIEVEMENT OF GATES STUDENTS 

Over the five years of the program, 78.8 percent of the fourth-grade 
and 72.9 percent of the seventh-grade Gates students attained their 
reading criterion on the CAT, D.R.P. or MAT. In addition, the average 
full-year gain was 16.4 N.C.E.s in the fourth grade and 14.0 N.C.E.s in 
the seventh grade. 

The criterion attainment and reading achievement of the Gates stu- 
dents in 1985-86 is different from that in the previous four years of 
the program. In 1985-86, for the first time, a larger percentage of the 
seventh graders than fourth graders attained their criterion. This de- 
gree of criterion attainment in seventh grade also is greater than for 
seventh-grade Gates students in any prior year. The 1985-86 average 
N.C.E. score gains, however, are lower than those observed in prior 
years and lower than those of the comparison group of 1980-81 students 
who would have been eligible had there been a program in that year. 

Further, patterns of achievement, which have been consistent during 
previous years, were different in 1985-86. In the past, students pro- 
moted in August and January had the highest pretest and posttest scores 
and showed the largest gains. During the current program year, pretest 
and posttest scores were approximately the same within each grade for 
students who met criterion in August, January, and April. These results 
may be a function of the change in the achievement test used. Students 
promoted in August and January had a posttest score on the CAT, while 
students promoted in April had a posttest score on the D.R.P, 
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The reading achievement gains of the Gates students who became 
Extension-eligible students continues to be a problem and shows indications 
of becoming even more problematic. In previous years, the gains of 
Extension-eligible students were low; in 1985-86, the gains were non- 
existent. 

The patterns of achievement by subgroups of the Gates population, were 
similar to those observed in previous years. In 1985-86, gains in reading 
achievement were less for the Resource Room students as well as for the LEP 
students than for the total Gates population. 
ACHIEVEMENT OF EXTENSION STUDENTS 

Achievement of the fifth- and eighth-grade Extension students continues 
to be problematic. The gains on the D.R.P. are educationally significant 
(according to the D.R.P. Manual), and approximately 13 percent more eighth- 
grade Extension students attained their criterion in 1985-86 than in 1984- 
85. Among the fifth-grade Extensin students, this increase was approxi- 
mately seven percent. Nevertheless, gains for the years are seven N.C.E.s 
lower than those observed in previous years for eighth grade and 3.3 
N.C.E.s lower for fifth grade. 
RECOMMENDATIONS 

O.E.A. recommends that program administrators note than even though 
more students are attaining their grade criterion, the gains made by these 
students are smaller than in previous years. It appears that the increased 
rate of criterion attainment was made by the full -year Gates students who 
met criterion in April, 1986. This increase in the percent of students 
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attaining criterion may be due to the new opportunity students had in 
April, 1986 to p^s s out of the Promotional Policy Program on either of two 
tests (the D,R,P, or the HAT) and, therefore, may rapresent an artifact of 
testing and not improvements in program outcomes. This observation is 
further substantiated by the lower gains observed for the 1985-86 
students. 

Despite apparent gains in criterion attainment, O.E.A. recommends that 
program administrators at the central and district levels examine their 
programs for adequacy of teacher preparation and curriculum materials, 
O.E.A. suggests that some students may, in fact, not be receiving 
appropriate services. 
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Appenaix A 



ACCOUNTING OF READING-ELIGIBLE GATES STUDENTS 
INCLUDED IN FULL-YEAR GAINS ANALYSES, 
1981-82 THROUGH 1985-86 
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Accounting of Reading-Eligible Stes Students Included In Full-Year 
Analyses, 1981-82, through 1985-86 hun-year 




Students originally 
held over 10,890 

Excused fro« 
the prograa 

Appeal (exception)^ Z69 2.5t 
Special cuiicaUon^ 34o 



Students actually 
held over 



13,847 



8.336 



12,063 



229 l.7t 
172 



768 5.6t 896 7.21 



Eligibles in gains 
analyses 



Mot in gains 
analyses 



10,281 
7,014 68.2 
3,267 31.8 



13,446 



7,618 



11,167 



6,235 



553 



5,682 



9,955 



7,436 



9,989 



6,983 



3.7X 



630 6.3t 766 I0.3t 954 g.6t 787 11.3t 



9,325 



6,670 



9,035 



6,196 



8,382 



844 10.lt 



7,538 



; UP,. ^.S.. 33. S.0 ,,»„.3 .,,3,^, 

'jn ,^„, .„ ^ ^ ^ ^^^^^^ ^^^^^ ^^^^ ^ ^^^^ 




and^the 
and 
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Appena.ix B 



STATISTICAL ADJUSTMENT OF PRETEST SCORES 
TO ACCOUNT FOR REGRESSION TO THE MEAN 
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STATISTICAL ADJUSTMENT OF PRETEST SCORES 
TO ACCOUNT FOR REGRESSION TO THE MEAN 



The equation usea to aajust pretest scores to account for the regression 
effect is taken from A;O.H. Roberts, "Regression Towaras the Mean ana the 
Regression Effect Bias," in New Directions for Testing ana Measurements , 
No. 8, 1980, (San Francisco, Jossey Bass), pp. 59-82/ 

The equation is: 



Jcs = Xs + 0^ (1 - 0) (Kg - Xs) 



This formula was usea with the appropriate CAT scale scores to correct 
April, 1984pretest means, where: 

CAT, Form C 



G raae 4 Graae 7 

Xcs = correctea pretest (scale score) mean 



of Gates-el igiDle stuaents * * 

Js = pretest (scale score) mean of 

Gates-el igiDle stuaents 378.0 459.0 

Xg = city-wiae (scale score) mean on 

pretest 450.8 533.4 

o = stanaara aeviation of pretest scale 

scores nationally 60.8 75.9 

s = stanaara aeviation of pretest scale 

scores city-wiae 51.0 59.2 

0 = coefficient of reliability ,86 .86 

The posttest was the D.R.P. 



*These values are computea on the following page. 

*^hese values are only for Gates-el igiole stuaents who were incluaea 
1n the full-year gains analyses. 
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Aojustment of Apnl, 1985 CAT for Stuoents' Gains 



from April. 1985 to Apnl. 1986 



Grace Four 



(N = 4,998, Js = 378.0) 

Xcs = 378.0 + (1 
(51.0)^ 

Xcs = 378.0 + 14.49 

Xcs =392.5 

Converts to 34 N.C.E.s 



.86) (450.8 - 378.0) 



Graoe Seven 



(N = 5,750, Xs = 459.0) 

Xcs = 459.0 .^25^' ^1 
(59. Z)'^ 

Xcs = 459.0 + 17.12 

Xcs =476.1 

Converts to 36 N.C.E.s 



.86) (533.4 - 459.0) 
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